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(54) Title: SUBTITLING TRANSMISSION SYSTEM 
(57) Abstract 

A method of simultaneously transmitting a video signal 
and encoded data representing graphic images is disclosed. The 
invention is particularly applicable for transmitting multilingual 
subtitles with a video program. The graphic images are rectangular 
regions within the active video area. They are transmitted in the 
form of bitmaps. The invention not only allows any character 
font or size to be displayed, but also the definition of e.g. a 
program provider's logo. The encoded data includes a time stamp 
to specify the time at which a subtitle is to be displayed. Preferred 
embodiments of the invention include the transmission of colour- 
look-up-table (CLUT) data and a compatibility code indicating a 
required minimum number of entries of said colour-look-up-table. 
For receivers with a CLUT having more entries than necessary, 
a map table is transmitted for mapping the pixel data width to 
the input width of the relevant CLUT. The method is applicable 
to Direct Video Broadcast systems wherein the video signal is 
MPEG2 encoded and the graphic images are accommodated in a 
private data stream of an MPEG2 Transport Stream. 



TS— 




212 



FOR THE PURPOSES OF INFORMATION ONLY 



Codes used to identify States party to the PCT on the front pages of pamphlets publishing international 
applications under the PCT. 



AT 


Austria 


GB 


United Kingdom 


MR 


Mauritania 


AU 


Australia 


GE 


Georgia 


MW 


Malawi 


BB 


Barbados 


GN 


Cuiaci 


NE 


Niger 


BE 


Belgium 


GR 


Greece 


NL 


Nethcj lands 


BF 


Burkina Faso 


HU 


Hungary 


NO 


Norway 


BG 


Bulgaria 
Benin 


IE 


Ire land 


NZ 


New Zealand 


BJ 


IT 


Italy 


PL 


Poland 


BR 


Brazil 


JP 


Japan 


FT 


Portugal 


BY 


Betaiui 


KE 


Kenya 


RO 


Romania 


CA 


Canada 


KG 


Kyrgystan 


RU 


Russian Federation 


CF 


Central Africarj Republic 


KP 


Democratic People's Republic 


SO 


Sudan 


CG 


Congo 




of Korea 


SE 


Sweden 


CH 


Switzerland 


KR 


Republic of Korea 


SI 


Slovenia 


CI 


Cbtc d'lvonr 


KZ 


Ksrtkhtttn 


SK 


Slovakia 


CM 


Cameroon 


U 


l_r tOC hit TWt^ TP 


SN 


Senegal 


CN 


China 


LK 


Sri Lanka 


TD 


Chad 


cs 


Czechoslovakia 


LU 


Luxembourg 


TG 


Togo 


cz 


Czech Republic 


LV 


Latvia 


TJ 


Tajikistan 


DE 


Germany 


MC 


Monaco 


TT 


Trinidad and Tobago 


DK 


Denmark 


MD 


Republic of Moldova 


UA 


Ukraine 


ES 


Spam 


MG 


Madagascar 


US 


United States of America 


n 


Finland 


ML 


Mali 


uz 


Uzbekistan 


FR 


France 


MN 


Mongolia 


VN 


Viet Nam 


GA 


Gabon 











WO 96/19077 

Subtitling transmission system. 



1 



PCT/IB95/0I118 



FIELD OF THE INVENTION 

The invention relates to a method of transmitting encoded data defining a 
graphic image in the form of a rectangular region within an active video area. The invention 
also relates to a method of receiving said graphic images, and a transmitter and receiver 
arranged to carry out said methods. 

5 

BACKGROUND OF THE INVENTION 

A prior art method of transmitting graphic images is generally known by 
the name •teletext". Hie World System Teletext standard has been adopted in many countries 
for the simultaneous transmission of textual and graphic information with a conventional 

10 analog television signal. The teletext system is particularly used for die transmission of 
subtitles associated with a television program. 

Meanwhile, the MPEG2 (ISO/IEC 13818) digital television standard will 
be applied worldwide for the broadcast of digital television programs to the end-user by 
satellite, cable, terrestrial networks, and by packaged media such as tape or disc. A 

15 multitude of integrated circuits for MPEG2 decompression and demultiplexing are on the 
market and will be applied in receiver decoders for the purpose of receiving and decoding of 
broadcast television programs or decoding of packaged (multimedia. 

The MFEG2 compression and multiplexing technology will provide the 
end-user a very large choice in programs (typically in the order of hundreds), which will 

20 require an easy-to-use "man-machine interface'. It is expected that the end-user will associate 
"digital* with high quality. High quality must therefore be manifest in all aspects of the 
service that will be provided. Typical of satellite broadcast is the large footprint of its 
reception area, which usually encompasses multiple language areas. Many of the television 
. programs to be broadcast will therefore also be targeted at multiple language areas, which 

25 requires multi-language subtitling capabilities. The attractiveness and readability of subtitles 
will have a major impact on the end-user's willingness to watch a television program in a 
language spoken other than his or her own. 
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The prior art teletext system is very limited in quality and performance. It 
is a character-based transmission system, in which coded data is transmitted to define which 
characters are to be displayed. The appearance of die subtitles (in terms of character font, 
spacing, graphic capabilities, colours) is fixed in hardware. Further, the present teletext 
5 system is not able to support complex character sets such as used in the People's Republic of 
China and Japan. The character-based system would require very expensive and complex 
character generators in case of supporting multiple character sets, or in case of a single but 
very complex character set The prior art teletext system thus provides only a basic solution 
for multi-language subtitling. 

10 

OBJECT AND SUMMARY OF THE INVENTION 

It is an object of the invention to overcome the above mentioned 

disadvantages of the prior art system. It is a further object to provide a universal solution for 

multi-lingual subtitling, in particular, far an MPEG2 environment. 
IS The method according to die invention is characterized in that the pixels 

farming said region are individually defined by the encoded data, the size and position of 

said region being included in the encoded data. Tae method renders it possible for the 

program providers to control (be appearance of the graphic images pixel-wise. The "Region 

based Graphics" system provides bitmapped rather than character coded regions overlaying 
20 the video signal. There are no restrictions as regards character size and character font, and 

there is a capability to overlay other graphics than text, e.g. the program provider's logo. 

In an embodiment, the encoded data further includes a time stamp 

representing the time at which said region is to be displayed. This allows multilingual 

subtitles to be transmitted in advance and to be displayed in exact synchronism with the 
25 video signal. In die prior art teletext system, the subtitles in the selected language are 

immediately displayed when received. More or less precise synchronization is possible only 

if all subtitles in the plurality of languages are transmitted within a restricted period of time. 

Subsequent encoded data for a given region may include a viability code 

to disable the display of said region. This allows for effectively erasing a graphic region. 
30 In further embodiments of the invention, the pixels of a region are 

encoded as entries of a colour-look-up-table. Map tables are provided for mapping n-bit pixel 

codes to a colour-look-up-table having more than 2 s entries. 

In an particularly advantageous embodiment, the encoded data is 

accommodated in a private transport dream according to the MPEC2 or a similar standard. 
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Packetizing the encoded data in PES packets of an MPEG2 private transport stream has the 
advantage of easy synchronizing the bitmaps to an flwvt»t~t MPEG video stream. 

BRIEF DESCRIPTION OF FIGURES 
5 Figs. 1-3 show examples of a display screen comprising one or more 

graphic image regions. 

Fig.4 shows a region with its relevant size and position parameters. 

Fig.5 shows a schematic diagram of a transmitter adapted to 
simultaneously transmit a video signal and graphic images according to the invention. 
10 Fig.6 shows a sc hem ati c diagram of a receiver for simultaneously 

receiving a video signal and graphic images according to the invention. 

Figs. 7 and 8 show embodiments of a colour generating circuit of the 
receiver shown in Fig.6. 

15 DESCRIPTION OF EMBODIMENTS 

The invention will now first be described in terms of the transmission 
format The syntax as described hereinafter is similar to the syntax used in the MPEG2 
(ISO/IEC 13818) standard. The { and } symbols of this syntax have been omitted as they are 
not significant to the readability. Also omitted are data elements which are not essential to 
20 the invention or merely serve the purpose of byte alignment 

The encoded data defining the graphic images is packetized in PES 
(Packetized elementary Streams) packets and accommodated in an MPEG2 Transport Stream. 
As prescribed by the MPEG2 standard, the PES packets comprise a header and payload. 

The header comprises a plurality of data elements, the following of which 
25 are relevant to the invention: 

* The element strtamjd is set to *10U 1101" so as to indicate that the graphic 
image is conveyed as private jlaiajtrtamj . 

* The element PES jacket Jcngth is set to a value such that each PES packet is 
aligned with an MPEG2 transport packet. 

30 * The element datajalignmentjndiaaor is set to T t indicating that the region 

data fields are aligned with the PES packets. 

* The PTS (Presentation Time Stamp) indicates the beginning of the presentation 
time of each (region related) data contained in this PES packet. The PTSs of 
regions within one multi page shall differ more than 40 msec. 
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The payload of PES packet defining the graphic images is denoted 
PES daxaJieldQ. Table I shows the syntax of this field. 

Table I 



PES_<teta_fMd(l 


8 uimsbf 


datsjkteniiOsr 


RbG~strsamjd 


16 bslbf 


strMmj9vtl_of_contpatibUitv 


3 bslbf 




1 bslbf 


for li-0;N;i++) 




region Jd 


8 uimsbf 


rao^^level.of.compatibUity 


3 bslbf 


rp0k>n_<tati_l6iiQlh 


16 uimsbf 


raoionjlataJteldO 





The following semantic definition applies to die data elements shown in this Table: 

* dmajdeimfier - Region based Graphics (RbG) data is identified by the values 
5 0x20-0x2F of the datajdemifier field* 

* EbGjtrtamJd - Identifies the Region based Graphics stream from which data 
is stored in this PES packet. 

* sneamjeveljofjcompaxibility - Indicates the minimum Colour-Look-Up-Table 
(GLUT) that is required to decode regions from this stream. The value 0x01 

10 refers to a 2-bit GLUT, the value 0x02 refers to a 4-bit GLUT, and the value 

0x03 refers to an 8-bit CLUT. 

* singlejxigejdisplay Jlag — If set to T, indicates that this RbG stream contains 
only one display page. 

* region Jd - Identifies the region from which the data is contained in the 
15 immediately following region Jam JieliQ. 

9 region_levelj>/jcompaxibiUry - Indicates the minimum CLUT that is required 

to decode this region. The values defined above apply. 

* region Jatajength - Specifies the number of bytes in the immediately 
subsequent region Jaia JUldQ the syntax of which is given in Table II. 

20 

Table II shows the syntax of the region jimaJxtldQ. 

The following semantic definition applies to the data elements shown in this Table: 

* regionjatajype - Identifies the type of region data. The value 0x10 refers to 
an intra region (I-region), the value 0x11 refers to region attributes (A-region), 

25 the value OxFF indicates a region to be delet ed. 
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Table D 



regtonjlanfieldl) 




regton_data_type 


4bsibf 


visibilityjlag 


1 bslbf 


horiiootaJ_sddrc*s_ftag 


1 bslbf 


verticsl_addmss_fl8g 


1 bslbf 


numberj>f_j>bcsis_ftag 


1 bslbf 


ruimber_ofJines_ftag 


1 bslbf 


if (horizomal_addressjla0- -'V) 




rsoion honzontat address 


16 uimsbf 


if <venk»faddmss_flag - -*1 '» 




region_verticaJ_sddress 


16 uimsbf 


if (ruirnbar_of_pixals_fiag - - 'V) 




number of jttxeis 


16 uimsbf 


if ImjmbeTof Jnesjtag - - ' V) 




number_of_Bnes 


16 uimsbf 


number _of_subregions 


4 uimsbf 


for (i - 0;i < mjmber_of_sub_regiona;i + + ) 




sub_regionjd 


4 bslbf 


nurnber_of_sub_reoionjtnes_flao 


1 bslbf 


dutfeQ 


1 bslbf 


sttributa_extension_flag 


1 bslbf 


coded_data_fiag 


1 bslbf 


coded_data_extension_flag 


1 bslbf 


if <munW_of_subj*gk>n Jines_ftag - -'V) 




number of sub region lines 


16 uimsbf 


if tdut_flao-«'vF 




colour Jook_up_tablejdataQ 




if (attribute_extenston_fiag - - # V) 




attribute_extenstonJength 


8 uimsbf 


ror u ■ i ;i<attnDute_extension_lengtn;i ♦ + 1 




reserved 


8 bslbf 


if <coded_dataflag- -'1') 




coded_sub_region_dataD 




if (coded_data_extension_flag - -T) 




coded_data_extensionJength 


8 uimsbf 


for (i«0;i<coded_teta_extensionjength;i + + ) 




reserved 


8 bslbf 



visibility Jlag - If set to f I\ specifies that this region shall be displayed as 
defined by the coded data for this region. If set to '0* this region shall be 
transparent. 

horizontal jiddress Jlag - Indicates the presence of the 
region Jiorizontal jiddress field. In I-regions this flag shall be *1*. 
verticaljaddress Jlag - Indicates the presence of the regionjtemcaljtddrzss 
field. In I-regions this flag shall be T. 

numberj>fj>ixels Jlag - Indicates the presence of the number j>fj>ixels field. 
In I-regions this flag shall be '1*. 
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15 



20 



25 



30 



numberjtfjines Jlag - Indicates the presence of the numberjefjines field. In 
I-iegicms this flag shall be '1*. 

region Jiorizontaljiddress - Specifies the horizontal address of the top left 
pixel of this region. Within the frame of 720*576 pixels only even pixels shall 
be addressed* The most left pixel of the 720 active pixels has index zero, and 
the pixel address increases from left to right The horizontal address shall be 
such that each pixel of the region is within the active 720 pixels. 
regionj*rtical_address - Specifies the vertical address of the top line of this 
region. The top line of the 720*576 frame is line zero, and the line index 
increases by one within the frame from top to bottom. The vertical address shall 
be such that each line of the region is within the 576 active lines. 
numberjrfjixels — Specifies the number of horizontal pixels of this region. A 
value of zero is forbidden. 

numberjyfjines - Specifies the number of vertical lines of this region. A value 
of zero is forbidden. 

number j>f_subjegions - Specifies the number of sub-regions within this 

region. A value of zero is forbidden. 

subjegionjd — Identifies the sub-region. 

number j)f_sub_region lines Jag - Indicates the presence of the 

number _pf_subj*gionJines field. 

clut Jlag - Indicates the presence of the colour Jookjip jiata JieldQ field, the 
syntax of which is given in Table m. 

attribute ^extension Jlag - If set to *1\ indicates the presence of the 
attribute ^extension length field and the associated reserved fields. 
coded jlaxa Jlag - Indicates the presence of the coded jubjegionjiaxaQ field 
which is defined in Table IV. 

coded jiatajatension Jlag - If set to T, identifies the presence of the 
coded jlatajaaensionjength field and associated reserved fields. 
number j>f_sub_regionJines - Specifies the number of lines in a sub-region. 
The total number of lines in all sub-regions within the region shall be equal to 
the number of lines in the region. 

axtributejaaensionjength - Specifies the number of immediately following 
reserved bytes. 
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• codedjaajxtauionjaiph - Specifies the number of immediately following 

reserved bytes. 

Table in shows the syntax of the colour _look_up_daiaJield(). 

Table m 



colour Jook_up_tabis_dataO 




numbt?jof_«ntries 


9 uimsbf 






2_bft_dutjlestinricM_fiag 


1 bsibf 


4_bit_ctut~d8StinaTjon~flaQ 


1 bsibf 


S.M.duTdastination'flao 


1 hslbf 


full_ranoa_flag 


1 bsibf 


entry_vahi 


8 uimsbf 


rf (fufiT range flag- -Tl 


Y_value " 


8 bstbf 


U value 


8 bsibf 


V>ah* 


8 bsibf 


T value 
«lse~ 


8 bsibf 


Y.vahie 


4 bsibf 


U_vatua 


4 bsibf 


V_vaJue 


4 bsibf 


T_vaiue 


4 bsibf 



5 The following semantic definition applies to the data elements shown in this Table: 

* number j>fjdmjipdaxes - Specifies the number of update cycles to be made. 

* number j>f_enmes — Specifies the number of CLUT entries that are updated. 
2JtijtojUsnrumonJlag - If set to *1\ indicates that this entry is intended 
for a 2-bit CLUT. 

10 • 4_bujdutjlesnnation Jlag - If set to T, indicates that this entry is intended 
for a 4-bit CLUT. 

* SJbitjdutjIestinarion Jlag - If set to T f indicates that this entry is intended 
for an 8-bit CLUT. 

* fidljangc Jlag - If set to T, indicates that the Y-value, U-value, V-value and 
15 T-value fields have full 8-bit resolution. If set to *0\ then these fields contain 

only the 4 most significant bits. 

* entry jwlue - Specifies the entry value of the CLUT. The first entry of the 
CLUT has the entry value zero. 

* Y_yaiue - The luminance output value of the CLUT for this entry. 

20 • Upvalue - The first chrominance output value of the CLUT for this entry. 



WO 96/19077 PCI7IB95A>1118 

8 

* Vjtalue — The second chrominance output value of the CLUT for this entry. 

* T_value — The transparency output value of the CLUT for this entry. A value 
of zero identifies no transparency. A value of 255 identifies full transparency* 
For all other values the level of transparency is defined by linear interpolation. 

5 

Table IV shows the syntax of the coded jvbjregwnjbxaO^ 

The following semantic definition applies to the data elements shown in this Table: 

* datajblockjength - Specifies the number of bytes of the 
coded jubjtgionJUxta field immediately following this field* 

10 * codingjnode - Specifies the applied coding mode. If set to W, identifying the 

presence of the modejextension Jield. 

* pixelsjnjnode - Specifies the number of pixels coded by a 2, 4 or 8 bit code 
as specified by the codingjnode field. 

* 2J>isjwde - A two bit code specifying the colour of a pixel. 
15 * 2jstuffJ>Us - two stuffing bits that shall be coded with *H\ 

* 4J>b_code - A four bit code specifying the colour of a pixel. 

* 4jaffJ>its - Four stuffing bits that shall be coded with f llH*. 

* 8J>is_code — An eight bit code specifying the colour of a pixel. 

* modejaaension - Identifying the applied extension mo d e . If set to *00\ 

20 identifies the presence of a command. If set to another value, identifies that run- 

length coding is applied. 

* lOJbitjw - Specifying the number of subsequent pixels with the colour 
identified by the immediately following 2Jbit_code field. 

* 8J>itjvn - Specifying the number of subsequent pixels with the colour 
25 identified by the immediately following 4J>is_code field. 

* d bitjun - Specifying the number of subsequent pixels with the colour 
identified by the immediately following 8Jbii_code field. 

* command Jengih - Specifies the number of bytes in this 

coded _subjregionjlma () field immediately following the commtmdjength field. 
30 If set to a value of 15, this number is not specified by the commandjength 

field. 

* longjength - Specifies the number of bytes in this coded jubjegionJtataQ 
field immediately following the longjength field. 
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Table IV 


PCI7IB95/01118 


coded_sub_region_dataO 




drajriodcjength 


16 uimsbf 


for <F- 1 <data_biockJength;i + + ) 




coding mode 


2 bslbf 


rf (codingjnodel - 'OCT) 




pixels in mode 


6 bslbf 


if (cod^gjnode- -'01') 




for (i - 1 ;i< (pixeis_in_mode + 3)/4;i ♦ + ) 




for (i-1;i<pixelsJn_mode;i + 4-) 




2_bit_code 


2 bslbf 


for i+ 1;i<(4 # ((pixels_in_fnode+ 3)/4)-pixets in mode);i++) 




2 stuff bits 


2 bslbf 


if Icodtna mods- -'10*) 




for (i-1;i < (pixels in mode* 1)/2;i+ + ) 




for 0 — 1 ;i < pixels in mods;i++) 




4_bftcode 


4 bslbf 


for 0- iTi < (2*((pixelsJn_mode + 1 )/2-pixels_rn_moda);i + + ) 




4 stuff bits 


4 bslbf 


if (codingjnode- -'IV) 




for (i — 1 ;i < pixels in mode;i + + ) 




8 bit code 


8 bslbf 


else 

modfi Axtnniiofi 

■ 1 IwWW WWW •••w* ■ 


2 bslbf 


if fmode extension! — '0OM 




if (mode extension — — '01 "I 




10 bit run 


10 uimsbf 


2 bit code 


2 bslbf 


if (mode extension — — ' 1 0*) 




8 bit run 


8 uimsbf 


4~bit~eode 


4 bslbf 


if (mode_extension --'1V) 




4_bft_run 


4 uimsbf 


8_bit_code 


8 bslbf 


else 




command Jength 


4 uimsbf 


rf (command Jength - - 1 5) 




long length 


1 6 uimsbf 


^fttitwiflfirt in 

W0%j+W99 iwiy ew 


SbsJbf 


if (command jd- - load_map_table) 




map_table_data<) 
else " ~ 




rf (command Jength! - 1 5) 




for (i - 1 ;i <command_length;i + + ) 






8 bslbf 


else 

for (i - 1 ;i < long Jength;i ♦ ♦ ) 




reserved 


8 bslbf 



command Jd - Identifies the type of command. The value 0x10 denotes Load 
Map Table. This command is followed by a mapjablejLuaO field which is 
defined in Table V. The value 0x11 denotes End of Line. 
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Table V shows the syntax of the map_table_dma(). 



Table V 



m»p_tabtodstaO 


8 uimsbf 


numb«r_of_tn tries 


for fl • ill <ruimber_of_emrias;i + + ) 


1 bslbf 


2_to_4jnap_table_flag 


2~to~8jnajTtaWe_flag 


1 bslbf 


4jD~8jnap3table_flag 


1 bslbf 


«mry_vatu* 


4 uimsbf 


m»p_tabte_output_v»lus 


8 uimsbf 



Tbe following semantic definition applies to the data elements shown in this Table: 

* number jofjtntries - Specifies the number of Map Table entries that axe loaded. 

* 2jojtjnapjable Jlag - If set to T, indicates that this entry is loaded into the 
S "2 to 4 Map Table. 

* 2joJSjnapjable Jlag - If set to T, indicates that this entry is loaded into the 
•2 to 8 Map Table. 

* 4jo_8jnapjable Jlag - If set to T, indicates that this entry is loaded into the 
•4 to 8 Map Table. 

10 * mapjablejype - Identifies the type of Map Table that is loaded. The value 

W denotes a "2 to 4 Map Table', the value f 10 f denotes a "2 to 8 Map 
Table", the value v ll* denotes a "4 to 8 Map Table". 

* entry jvalue - Specifies the entry of the Map Table. The first entry of the Map 
Table has the entry value zero. 

15 * map table oiupux_yalue — Specifies the output value of the Map Table for this 

entry. 

The above syntax and semantics will now be elucidated in a more 
common language. The PESjlaia JieldQ (Table I) specifies at least erne "page" comprising 

20 one or more regions. Figs. 1-3 show some examples of an active video area comprising one 
or more regions. Rg.l shows a program provider's logo A and a current subtitle B. Fig.2 
shows an additional subtitle C for hearing impaired viewers (hearing impaired people often 
need additional explanatory text such as "audience applauds", "swelling music"). Fig.3 
shows the logo A and a subtitle Bi, the index i indicating a selectable one from a plurality of 

25 languages. 
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The data for each region is accommodated in the region jiata JieldO 
(Table II). This field specifies the size and position of the respective region. Referring to 
Fig.4, die position of a region is denoted (x,y), x being specified by 
regionjiorizomaljiddress and y by regionjvrticaljaddress. These position parameters can 
5 be changed in time which allows for scrolling, by retransmitting encoded data for a given 
region (specified by its region Jd) without retransmitting pixel codes and colour-look-up-table 
data. The size of the region is p*n, p and n being specified by number j>fj>ixels and 
number jofjines, respectively. Each region comprises up to 16 sub-regions. In Fig.4, two 
subregions comprising n, and n? lines, respectively, are shown. The region jLua JieldO 

10 specifies colour-look-up-table data and/or pixel data for each subregion. 

Not all parameters and data mentioned above are compulsory. Their 
pr e sen ce is indicated by respective flags. This allows already defined regions and/or 
subregions to be updated or modified by referring to their region Jd and/or subjrgionjd, 
respectively. For example: the position of a region can be modified (which allows scrolling); 

15 an already displayed region can be made invisible; or a subregion can also be referred to by 
an other region and thus displayed elsewhere on screen without all image data being 
retransmitted. 

The colour-look-up-table is loaded or updated by the 
colour JookjpjlataO field (Table HI). This field specifies which luminance, chrominance 

20 and transparency output values are to be stored in which entries of the CLUT. The 

transparency defines to what extent the video and graphics pixels are to be mixed on screen. 

The pixel data for a subregion (or for a whole region is the number of 
subregions is 1) is accommodated in the field coded jub ^region JLqiq() (Table IV). The 
pixels are defined as entries of the colour-look-up-table. The pixel values can be defined in 

25 2 V 4 or 8 bits. Three coding modes are provided. In a first coding mode (coding jnode is 
'01*, *10\ V H*)> consecutive pixels are individually specified. The number of pixels thus 
transmitted is defined by pixelsjnjnode. In a second coding mode (coding jnode is W, 
coding mode ^extension is '01*, '10*, *11*), runs of consecutive identical pixels are run- 
length encoded. The first and second coding modes can be mixed within a region. In order to 

30 minimi^ decoding complexity care has been taken that the run-length coded data is byte 
aligned. Each run-length code comprises two bytes: 2 bits for the coding mode, 2 bits for the 
coding mode extension and 12 bits for the run-length code. The latter defines a 10-bit run of 
2-bit pixels, an 8-bit run of 4-bit pixels, or a 4-bit run of 8-bit pixels, as specified by the 
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coding mode extension. A third coding mode (coding jnode is '00', coding jnodejsxtension 
is *00*)t sptn fi »* commands such as map tables and end-of-iine indicators which work on the 
run-length decoded data. The concept of map tables is disclosed in more details below. 

It is envisaged thai existing On-Scieen-Display chips are used for 
5 displaying the graphic images. Some of these chips allow for 2 bits per pixel, others for 4 
bits per pixel or 8 bits per pixd. A potential problem arises if the pixels within a region are 
encoded using a number of bits per pixel different from the colour-look-up-table's input 
width. In order to solve this problem, the concept of map tables has been introduced. Map 
tables also increase the coding efficiency. They allow pixels to be en c od e d in less bits per 
10 pixel than the entry-width of the colour-look-up-table. These aspects will now be elucidated 
by way of an example. Suppose a (sub)region in which only four different colours are used. 
The respective colours are stored, in terms of luminance and chrominance, in four entries of 
a CLUT. The pixel data for said region is encoded in 2 bits/pixel. The decod er , however, 
may have a 4-bit CLUT and thus require four bits per pixel. Now, the map table provides 
15 the mapping of the transmitted 2-bit/pixel codes to a 4-bit CLUT entry. The map table thus 
informs a decoder which four of the 16 entries are to be used. Whether or not the 2-bit/pixel 
coding indeed improves the efficiency depends on the overhead to change the coding mode or 
the map table. Thus, map tables are provided for mapping n-bit/pixel coding to an m-bit 
CLUT, n and m being different numbers. In view of this elucidation of the concept of map 
20 tables, the mapjablcjtataO field (Table V) needs no further explanation. For each entry 
value of the map table, an output value is defined. 

Fig.5 shows a schematic diagram of a transmitter according to the 
invention. The transmitter is adapted to simultaneously transmit a video signal and graphic 
images The transmitter receives a baseband video signal in YUV format and an associated 
25 audio s ig nal AUD and comprises a MPEG2 video encoder 101 and MPEG2 audio encoder 
102 for mending said video and audio signal into packetized elementary streams PSV and 
PSA, respectively. The transmitter further comprises an editing terminal 103 for editing 
subtitles and/or other graphic images such as a program provider's logo. The editing terminal 
is coupled to a processing and storage device 104. A plurality of subtitles associated with the 
30 relevant television program, possibly in a plurality of different languages, is assumed to be 
stored in the processing and storage device in advance. The video signal is also applied to a 
time generator 105 which applies a time t to the processor and storage device 104. The time 
t can be the time which has elapsed since the start of the relevant television program, or can 
be a time code arairia tH with the program and stored on the same tape as the television 
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signal. In MPEG systems, the time t will correspond with MPEG's System Clock Reference 
SCR. The proces so r and storage device compares the time t with a time stamp as yyi fl twf 
with each stored subtitle. When a subtitle is due, the relevant data is applied to an r™re*ing 
and packetiier 106 which forms a packrrircd elementary stream PSG defining encoded data 
5 according to the transmission format discussed hereinbefore. The packrti?fd elementary 
streams PSV t PSA and PSG are multiplexed by MPEG multiplexer 107 so as to form a 
transport stream TS. The transport stream is transmitted to a receiver via a transmission or 
storage medium (not shown). 

Fig.6 shows a schematic diagram of a receiver according to the invention. 
10 The receiver comprises an MPEG2 demultiplexer 201 which receives the MPEG2 transport 
stream TS and selects the packeti?ed audio stream PSA t video stream PSV and rncodfd 
graphics data PSG together constituting a selected television program. The audio stream is 
decoded by an MPEG2 audio decoder 202. The decoded audio signal AUD is applied to a 
speaker 203 for reproduction. The video stream is decoded by an MPEG2 video decoder 204 
IS to obtain the decoded video signal in YUV format for display on a display screen 205. The 
encoded graphics data stream is applied to a data parser 206. This circuit parses the data 
accommodated in the PES packets PSG and provides the following output signals: 

the presentation time stamp PTS accommodated in the PES header, indicating 
the time at which the region is to be displayed; 
20 - the value of the visibility Jlag (Table II), denoted VF; 

the encoded pixel data as defined by the fields njbitjtode in Table IV 

(n =2,4,8); 8-bit pixel codes are denoted PX8, 4-bit pixel codes are denoted 

PX4, and 2-bit pixel codes are denoted PX2; 

the position (x,y) and size (p,n) of the currently received region as defined by 
25 the fields region Jwrizontal address, region jvnicaljiddress, 

number j>/j)ixels, and numberjyfjines, respectively (Table n, Fig.4); 

map table data denoted MTD defined by the field mapjablejLuaO (Table V), 

(this signal is not used in the present embodiment); 

a signal denoted LoC derived from the field region Jevelj)f_compcaibiliry 
30 (Table I), and 

colour-look-up-table data denoted CTD, defined by the field 

colour Jook _upjablejtasa() (Table HI). 
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The pixel data PXn (n«2,4,8) received from die parser 206 is stored 
(when run-length encoded via a run-length decoder 207) in a pixel memory 208, the relevant 
memory locations being determined by position (x,y) and size (p,n) received from the parser. 
The pixel memory 208 is read out by display timing signals (not shown). The pixels PX2, 

5 PX4 or PX8 (a region may comprise both 2-, 4- and 8-bit pixels, but only one of them is 
active for die duration of a pixel) are thus applied to a colour generating circuit 209 in 
synchronism with the received video signal. 

The colour generating circuit 209 comprises a colour-look-up-table 2092. 
In this embodiment, said CLUT is assumed to have four entries only, i.e. its input is 2 bits 

10 wide. The CLUT holds, for each entry, an output signal denoted OSD in terms of an 8-bit 
luminance value Y and 8-bit chrominance values U and V, and an 8-bit transparency value 
T. The relevant YUV and T values are stored in the CLUT in response to the colour-look- 
up-table data CTD received from the parser. 

As the colour-look-up-table 2092 has a 2-bit input, the 2-bit pixels PX2 

15 are directly applied thereto. The 4-bit pixels PX4 and 8-bit pixels PX8 are applied to the 
CLUT via a 4-to-2-bit reduction circuit 2093 and an 8-to-2-bit reduction circuit 2094, 
respectively. In a ample embodiment, said reduction circuits merely discard the least 
significant input bits. In an alternative embodiment, the most significant bit is unaltered (or 
some of the most significant bits are OR-ed) to obtain the first output bit whereas less 

20 significant input bits are OR-ed to obtain the second output bit 

The reduction circuits 2093 and 2094 allow higher level CLUT data (>2 
bits) to be compatibly decoded by lower level CLUTs (2 input bits). However, a program 
provider may prevent regions defined by more than 4 colours to be decoded by 2-bit CLUTs. 
This is achieved by a switch 2091 controlled by the signal LoC received from the parser, i.e. 

25 by the field regionJevelj>f_companbiliry (see Table I). If the switch is in the position 
denoted 4 or 8 fi.e. the program provider requires at least a 4-bit CLUT or 8-bit CLUT, 
respectively) the region will not be displayed. Only if the switch is in the position denoted 2 
(i.e. the program provider accepts that pixels are displayed in four colours only), the pixel 
data is indeed applied to the CLUT. 

30 In response to the 2-bit pixel codes, the CLUT 2092 delivers the display 

output signal OSD and the transparency signal T. The output signal OSD is applied to a 
mixing stage 210 in which it is mixed, on a pixel-by-pixel basis, with the video signal YUV. 
The transparancy signal T determines the amount of transparency of graphic pixels. If T has 
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the value 0, the video signal is blanked completely and the graphic pixels are not affected. If 
T has the value 255 (i.e. 100%), the relevant graphic pixel is fully transparent. For other 
values of T, die level of transparency is linearly interpolated. This allows the video signal to 
remain visible through the graphic image. 
5 The combined signal is then displayed on the display screen 205. 

However, whether or not the graphic image is indeed displayed depends on a switch 212 
which is controlled by a display enable circuit 213. This circuit receives the presentation time 
stamp FTS and the visibility flag VF from the parser, as well as (not shown) the current time 
t as conveyed by MPEG's System Clock Reference. The display enable circuit 213 is adapted 

10 to enable the display of the graphics region as soon as t>FTS and to disable its display when 
the visibility Jlag for the relevant region is set to *0*. 

Fig.7 shows a further embodiment of the colour generating circuit 209 of 
the receiver. In this embodiment, the CLUT 2092 of the receiver has a 4-bit input, Le. the 
CLUT comprises sixteen entries. Now, the 4-bit pixels PX4 are directly applied to the switch 

15 2091 whereas the 8-bit pixels PX8 pass through an 8-to-4-bit reduction circuit 2096 which, 
for example, discards the four least significant input bits. The 2-bit pixels pass through a 2- 
to-4-bit map table 2095, in fact being a memory having four 4-bit locations. The map table 
2095 maps the 2-bit pixels into 4-bit CLUT entries as specified by map table data MTD 
received from the parser. The concept of map tables has already been described before. If 

20 the switch 2091 is in the position denoted 8 (i.e. the program provider requires at least an 
8-bit CLUT), the region will not be displayed. If the switch is in the position denoted 2 or 4 
(i.e. the program provider accepts that 8-bit pixels are displayed in sixteen colours only), the 
pixel data is indeed applied to the CLUT. 

Fig. 8 shows an embodiment of the colour generating circuit 209 in which 

25 the CLUT 2092 has an 8-bit input, i.e. the CLUT comprises 256 entries. Now, the 8-bit 
pixels PX8 are directly applied to the switch 2091 whereas the 2-bit pixels PX2 and 4-bit 
pixels PX4 pass through a 2-to-8-bit map table 2097 and 4-to-8-bit map table 2098, 
respectively. The map table 2097 maps 2-bit pixels into 8-bit CLUT entries as specified by 
map table data MTD received from the parser. The map table 2098 does the same for 4-bit 

30 pixels. Strictly speaking, the switch 2091 is redundant in this embodiment, as this receiver 
deals with all transmissions regardless of the transmitted CLUT requirements. 

As explained above, reduction circuits (2093, 2094, 2096) allow higher 
level CLUT data to be compatibly decoded by lower level CLUTs, whereas map tables 
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(2095, 2097, 2098) allow lower level CLUTs to be dynamically mapped in higher level 
CLUTs. Table VI summarizes the various options and w"ti c*tr? (he various levels of 
performance. 



Table VI 



2-bit CLUT 


0 


0 i 


0 • - : 


4-bit CLUT 


0 


+ j + + 


+ j ++ 1 


8-bit CLUT 


0 


+ j + + 


+ + j + + + ! + + + + 




LoC«2 


LoC = 2 | LoC=4 


LoC =2 | LoC=4 j LoC =8 


2-bit 
data 


4-bit 
data 


8-bit 
data 



Table VI shows three rows indicating the performance of receivers having a 2-bit, 4-bit and 
5 8-bit CLUT, respectively. Three columns indicate whether a region comprises 2-bit, 4-bit or 
8-bit pixels, respectively. LoC denotes the level of compatibility included in the encoded 
data. 

If a region is defined in 2 bits/pixel only, four colours can be displayed 
(performance denoted 0) irrespective of whether the CLUT input is 2, 4 or 8 bits wide. It 

10 does not make sense for a program provider to require, for such a region, a better than 2-bit 
CLUT. This is in dicate d by LoC=2 in the respective column. 

If a region comprises 4-bit pixels, the program provider may allow such a 
region to be displayed using a 2-bit CLUT (LoC =2). Said 2-bit receivers display the region 
in four colours (performance denoted 0), whereas 4-bit and 8-bit receivers display 16 colours 

IS (performance denoted +)• However, the program provider may also require the receiver to 
have a 4-bit CLUT (LoC =4). In that case, a 2-bit receiver can not display the region 
(performance denoted *) whereas 4-bit and 8-bit receivers will display 16 colours 
(performance denoted ++). 

If a region comprises 8-bit pixels, the program provider may allow such a 

20 region to be displayed using a 2-bit CLUT (LoC =2). In that case, 2-bit receivers display 4 
colours (performance denoted 0), 4-bit receivers display 16 colours (performance 
denoted +), and 8-bit receivers display the full pallet of colours (performance denoted ++). 
The program provider may also require the receiver to have at least a 4-bit CLUT (LoC =4). 
In that case, 2-bit receivers can not display the relevant region (performance denoted -), 4-bit 

25 receivers display 16 colours (performance denoted + +), and 8-bit receivers display the full 
pallet of colours (performance denoted +++). If the program provider requires the receiver 
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to have an 8-bit CLUT (100=8), neither 2-bit nor 4-bit receivers can display the relevant 
region (performance denoted -) f whereas 8-bit receivers have an optimal performance 
(denoted ++++). The latter perfonnace is optimal because the program provider does not 
need to take die restrictions of lower level receivers into account 
5 Summarizing the invention, a method of simultaneously transmitting a 

video signal and encoded data representing graphic images is disclosed. The invention is 
particularly applicable for transmitting multilingual subtitles with a video program. The 
graphic images are rectangular regions within the active video area. They are transmitted in 
the form of bitmaps. The invention not only allows any character font or size to be 

10 displayed, but also the definition of e.g. a program provider's logo. The encoded data 
includes a time stamp to specify the time at which a subtitle is to be displayed. Preferred 
embodiments of the invention include the transmission of colour-look-up-table (CLUT) data 
and a compatibility code indicating a required minimum number of entries of said colour- 
look-up-table. For receivers with a CLUT having more entries than necessary, a map table is 

15 transmitted for mapping the pixel data width to the input witdh of the relevant CLUT. The 
method is applicable to Direct Video Broadcast systems wherein the video signal is MPEG2 
encoded and the graphic images are accommodated in a private data stream of an MPEG2 
Transport Stream. 
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1. A method of transmitting encoded data defining a graphic image in the 

form of a rectangular region within an active video area, characterized in that the pixels 
forming said region are individually defined by the encoded data, the size and position of 
said region being included in the enc oded data, 
S 2. A method as claimed in claim 1, wherein the encoded data for said region 

further includes a time stamp representing the time at which said region is to be displayed. 

3. A method as claimed in claim 2, wherein subsequent encoded data for a 
given region includes a visibility code to disable the display of said region. 

4. A method as claimed in claim 1 9 wherein a region comprises a plurality of 
10 subregions including an identifier code to allow a graphic image defined by said subregion to 

be referred to by a further region. 

5. A method as claimed in claim 1, wherein the pixels for a region are 
encoded as entries of a colour-look-up-table, the encoded data including a compatibility code 
indicating a required minimum number of entries of said colour-look-up-table. 

15 6. A method as claimed in claim 5, wherein the pixels are defined by n-bit 

pixel data, the encoded data further including a map table for mapping said n-bit pixel data to 
a colour-look-up-table having more than 2" entries. 

7. A method as claimed in any of the preceding claims, wherein the encoded 
data is accommodated in a private transport stream according to the MPEG2 standard or 

20 extension thereof. 

8. A method of receiving encoded data defining a graphic image in the form 
of a rectangular region for display within an active video area, characterized by the steps of 
decoding said encoded data into individual pixels constituting said region, decoding the size 
and position of said region, storing said pixels, and generating display signals representing 

25 said pixels. 

9. A method as claimed in claim 8, further comprising the step of decoding a 
time stamp inchrtrd in said encoded data, and generating the display signals from a time 
represented by said time stamp. 
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10. A method as claimed in claim 9, further comprising the steps of decoding 
a visibility code included in subsequent encoded data for a given region, and rfitahiing the 
generation of said display signals for said region in response to said visibility code. 

11. A method as claimed in claim 8, wherein a region comprises a plurality of 
5 subiegions including an identifier code, comprising the step of including said subregion in a 

further region referring to said subregion. 

12. A method as claimed in claim 8, wherein the pixels for a region are 
encoded as entries of a colour-look-up-table and the encoded data includes a compatibility 
code indicating a required minimum number of entries of said colour-look-up-table, 

10 comprising the step of disabling the generation of display signals if the required number of 
entries is less than available. 

13. A method as claimed in claim 12, further comprising the steps of 
receiving map table data, and storing said map table data in a map table for mapping n-bit 
pixel data to a colour-look-up-table having more than 2* entries. 

15 14. A method as claimed in any of claims 8-13, wherein the video signal is 

encoded according to an MPEG2 standard or extension thereof, and wherein the encoded 
data defining the graphic image is accommodated in packets of an MPEG2 private transport 
stream. 

15. A transmitter comprising means for transmitting encoded data defining a 
20 graphic image in the form of a rectangular region within an active video area, char acteri zed 

by means for encoding the individual pixels forming said region and including the size and 
position of said region in the encoded data. 

16. A receiver coupled to a display screen for receiving encoded data defining 
a graphic image in the form of a rectangular region within an active video area, 

25 characterized by means for decoding said encoded data into individual pixels constituting said 
region, means for decoding the size and position of said region, and display means for 
storing and displaying said pixels. 

17. An image signal comprising encoded data defining a graphic image in the 
form of a rectangular region within an active video area, characterized in that the pixels 

30 forming said region are individually defined by the encoded data, the size and position of 
said region being included in the encoded data. 

18. A storage medium on which an image signal is stored comprising encoded 
data defining a graphic image in the form of a rectangular region within an active video area, 
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characterized in that the pixels forming said region are individually defined by the encodfd 
data, the size and position of said region being included in the encoded data. 
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